Australian Organizations are Fascinated With Copilot for Microsoft 365, But Will They Avoid The “Gotchas”?

The interest in Microsoft Copilot is high, with many major Australian organizations signing on to be part of Microsoft’s early access and pre-launch testing process.

According to Microsoft, 70% of Copilot users say they are more productive, and overall, there’s a 29% increase in speed with tasks like searching, writing and summarising. Copilot use is only going to accelerate, as Microsoft and its partners embed the AI product directly into PCs and processes.

Gartner recently published a series of “gotchas” with Copilot, and these are things Australian organizations need to consider in order to fully think through implementations and to benefit from what Copilot offers.

What are Microsoft Copilot’s gotchas?

Gartner’s list of gotchas across four categories essentially highlights where an implementation of Copilot might fail to deliver, or “surprise” the company with challenges that hadn’t been anticipated. The gotchas are grouped into four categories: administration, security, information governance and user experience.

Administration

Organizations can be exposed to greater risk and cost if:

• They fail to consider the proper configuration settings.
• The reporting tools lack granularity.
• The options for extending Copilot and managing costs are not well understood.

Security

Poor management of the use of Copilot can result in an increased risk of overshared information becoming exposed. Also, there are new attack surfaces that need to be monitored.

Information governance

Without first developing the ability to prioritise content sources, mitigate the risk of content and app sprawl, and manage the new retention and compliance challenges introduced by Copilot, organizations may not get the quality responses out of Copilot they were expecting.

User experience

The assumption that people will embrace Copilot and start using it as though they were comfortable with it seems to be misguided, and many organizations report a higher-than-expected change management effort.

Due diligence

The sum of these gotchas indicate that Australian organizations need to first fully canvas what Copilot brings to the business, how it will be used, and who in the organization will have access to it and why they need it.

Without that due diligence and then strategic deployment, it’s likely the organization will be surprised by something unexpected from the AI, resulting in inefficiencies, expense or even a lowering of productivity.

DOWNLOAD this AI Resource Pack from TechRepublic Premium

Australia’s position on the AI adoption curve

Australia isn’t significantly different from the rest of the world in terms of the uptake of Copilot or the potential gotchas, Gartner Senior Director Analyst Nate Suda said in an interview with TechRepublic.

What these gotchas essentially point to is that Copilot and other AI applications are not that dissimilar to digital transformation a few years ago, or cloud computing a decade before that. In each instance, there’s a recognition of the value of the technology but not an understanding of the costs and the value the technology will deliver.

SEE: Australia Is Adapting Fast to a Generative AI World (TechRepublic)

“There’s a lot of pressure on everyone in the C-suite. It’s not just a CIO thing. If you are a CEO, you’re under pressure from your shareholders to explain what the company is doing with AI,” Suda said.

“Most people understand that there is an enormous potential here, and the conversation has been going on for a year now. So for many organizations, it’s a matter of getting going now. 2023 was a year where it was all new, and we could marvel and wonder about it and play around with it a little. The conversation that we’re seeing in 2024, meanwhile, is more along the lines of ‘we’ve been playing for a year, now we don’t want POCs. We don’t want MVPs. We want to start seeing the ROI.”

Copilot is seen as a strong opportunity to start to move forward on AI projects; however, it’s chasing that ROI that can lead organizations to fall for one or more of the Copilot gotchas.

How are Australian organisations using Copilot?

Microsoft has engaged deeply with Australian businesses through the development of Copilot. In September 2023, Microsoft announced that through its invitation-only Early Access Program, a broad suite of organisations were using Copilot, including AGL, Data#3, Bupa, NAB, Powerlink Queensland, Rest Super and Suncorp.

In April 2024, Microsoft announced several Australian organisations, including Australian Super, Powerlink Queensland and TAL, had adopted Copilot specifically to strengthen their cyber security capabilities. The ROI here, according to Microsoft, is that experienced security analysts are 22% faster with Copilot and 7% more accurate.

Copilot is also being cited as a tool that can help upskill less experienced cyber security professionals and teams, and address the ongoing skills shortage in security.

SEE: Will Australia Ever Dig Itself Out of the Cybersecurity Skills Shortage?

The public service is also now moving on Copilot, with more than 50 agencies announcing they have commenced a six-month trial of Copilot. The program involved more than 7,400 public servants using the technology.

This project is being coordinated by the Digital Transformation Agency, again highlighting the parallels that AI and transformation have in terms of their impact on organisations as they adopt solutions.

As the CEO of the Digitally Transformation Agency, Chris Fechner is quoted as saying at the time: “The APS and the DTA will keep looking for opportunities to lift our Digital Government performance as evidenced by our inaugural OECD ranking and strive to improve on it in future years for the benefit of all Australians.”

“These goals go hand in hand with the Government for the Future mission that strives to take advantage of emerging technologies to deliver secure, ethical, and modern data and digital technologies by 2030.”

Will Microsoft Copilot deliver?

Now that businesses are moving beyond the trial of Copilot and starting to integrate it into their processes, it remains to be seen whether these projects will deliver, or whether organizations will fall into the gotchas.

Digital transformation was notoriously difficult when it was the “big thing” that CIOs were throwing money at, often without the proper due diligence, simply because it was expected that IT would be investing in digital transformation. The consequence of that was, according to data published in 2020, 70% of projects failed to deliver value.

Organizations need to be strategic about how they review, deploy and then measure the outcomes that they get from their Copilot investments.